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ABSTRACT 



This research brief describes how to conduct a 
cost-effectiveness analysis. The first task is to find out if there are 
differences in the costs of operating program alternatives through the 
following steps: (1) describing each program alternative and its components 

and potential benefits; (2) determining the costs of the alternatives by 
identifying cost factors, deciding on the data entry method, entering the 
cost data, and calculating the program costs; (3) determining cost savings; 
and (4) calculating and analyzing program costs. The second task is to 
determine whether there are differences in the effectiveness and cost 
effectiveness of feasible program alternatives. The third task is then to 
calculate and analyze cost-effectiveness ratios before moving to the final 
step of making decisions with cost analysis results. In the final analysis, 
all educational decisions, from the student- teacher interaction to the 
formulation of national policy, are cost utility data. (SLD) 
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HOW TO CONDUCT A COST-EFFECTIVENESS ANALYSIS 



The purpose of this research brief is to describe how to 
conduct a cost-effectiveness analysis. 

How do you conduct a cost-effectiveness analysis? 

Three tasks must be completed to complete a cost-effective- 
ness analysis. First, the costs must be determined for 
alternative ways to deliver the program. Next, the effec- 
tiveness must be determined. Finally, the cost-effectiveness 
between the program alternatives is established. 

TASK I: Are there any differences in the cost of 
operating program alternatives? 

Every cost analyses begins by describing the program, 
determining the costs and cost savings of the program 
alternative, and then analyzing the costs in a decision 
framework. 

Step 1: Describe each program alternative and its 
components and potential benefits. The description acts as 
a guide to determining costs and intended effectiveness of 
the project. 



ascertain their costs. Program costs are generally measured 
by three approaches: the estimated (aggregate) approach,’ 
the detailed (ingredients) approach, and the project evalua- 
tion approach. Only the first approach will be discussed. 

Estimation Method . In general, most estimates focus on 
costs associated with hardware and assume other costs 
are the same as normal school operations. This practice 
tends to underestimate the cost of instructional use of 
technology by focusing on what might be the least 
expensive item, the hardware. However, estimates of 
cost data can be reasonably accurate if internal data are 
good and estimates are made carefully. 

The unstated assumptions of the cost estimation method 
- usually accounts for wide variance in cost estimates. 
Inaccuracies can also be attributed to faulty estimates in 
the following cost factors: 1) operating costs, telecom- 
munications and courseware development, 2) rate of 
technology use, 3) number of students served per day, 
week, etc., 4) the length of the school day and year, 

5) amount of time and number of sites at which the 
system will be. used, and 6) the life span of system 
hardware (3-5 years) and courseware (10 years for basic 
skills software and less for other software). 



Step 2: Determine the costs of program alternatives by 
identifying cost factors, deciding on the data entry method, 
entering the cost data and calculating program costs. 

1. Identify cost factors . Cost analysis begins with 
identifying the cost factors required for a program, project, 
alternative or intervention. Cost factors include computer 
hardware, software, program development, direct person- 
nel, facilities, training and evaluation. 

Direct personnel refers to personnel required to operate the 
program. In addition, it allows you to identify personnel 
required in each of the other cost factors (indirect person- 
nel). An example would be, personnel costs related to 
developing software, personnel required to deliver training. 

2. Decide on the Method to Enter Data . Once cost 
factors have been identified and stipulated, it is necessary to 



There are several ways to approach cost estimation. 
Equipment can be estimated by price of equipment 
currently available. Non-equipment costs can be 
estimated at three times the investment in equipment. 

The value of facilities can be determined by estimating 
their lease value. The annual value of facilities and 
equipment can be estimated through a relatively simple 
approach that takes into account depreciation and interest 
foregone by remaining capital investment. 

A typical estimation scenario is: costs may be estimated 
by examining the prices of high quality equipment 
currently available. For example, in 1992, the cost of an 
IBM compatible computer with monochrome monitor, 

140 megabyte hard disk drive, a modem and a printer 
was $ from vendors on a state contract. To 

provide one work station with this equipment for every 
25 students would cost roughly $ not including 

software, furniture, local area network, staff training, etc. 
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In general, the following rules of thumb can be used in 
cost estimation: 

Market Value . The value of a cost factor is its market 
value and/or its market price. Market price is particu- 
larly useful when costing out equipment, materials and 
utilities. In the case of personnel, market value may be 
ascertained by determining what the costs would be for . 
hiring a particular type of person. Such costs must 
include not only salary, but also fringe benefits and other 
employment costs paid by the employer. 

Lifetime costs . Lifetime costs are deferred costs spread 
out over a period of time. As true cost declines, the more 
they can be differed over time. Lifetime costing accounts 
for the expected useful life of each cost factor. For 
example, lifetime costs of computer hardware (8 years) ' 
and software (4 years) were obtained by surveying 
experts in the field of educational microcomputing. 

a. Lease Value . The value of facilities can be 
determined by estimating their lease value.. 

b. Detailed Method . The detailed approach is the most 
accepted approach to determining costs. Even though 
it is the most time consuming method, the detailed 
approach is. also the most accurate method because it 
requires that each cost factor be broken into its 
separate components, it provides cost data on each 
(determining annual and deferred costs), and sum . 
these to a total. 

c. Project Evaluation Method . The project evaluation 
method provides information to continue the project, 
adjust the project,- or replicate the project. It permits 
you to: 1) evaluate the actual costs against the 
projected costs used when justifying the purchase 
decision, 2) appraise actual effectiveness of the 
project on a number of indicators for which data was 

- collected during the evaluation period, and 3) make * 
future decisions regarding the cost-effectiveness of 
. the project on a solid foundation. 

3. Enter Cost Data . Once the data entry method has ' 
been decided, each factor is costed, and summed to obtain a 
total cost. This step is most important in developing 
accurate cost analyses. It is strongly affected by the 
availability and reliability of cost data. Three principal 
sources of information — reports, observations and 
interviews - — can be used to assure the accuracy of the data 
by comparing findings from each source and reconciling 
differences. 



Levin (1981) admonishes educators to be systematic rather 
than casual. Data from accounting and budgetary reports 
must be used selectively and appropriately. This data 
cannot be relied upon for all cost factors. For example, they 
omit completely or understate cost savings such as: the cost 
of volunteers and other donated resources, or capital 
improvements charged during the year of their purchase 
when they have a life of 20-30 years. • 

Step 3: Determine Cost Savings. Are there cost savings 
associated with the project? Cost savings can be: 

1) predicted prior to adoption of an alternative, 2) collected 
during and after completion of an adopted alternative, or . 
3) analyzed and evaluated prior to replication. 

The person who is designing the project determines what 
cost savings are expected to occur as the result of a specific 
expenditure. For example, is equipment or personnel being 
donated? What tax savings can be generated? How much 
fuel savings occurs? 

Step 4: Calculate and Analyze Program Costs. The total 
cost of a project is obtained by summing the costs for each 
factor and subtracting the cost savings. 

Once cost data are generated, the problem of appropriate 
analysis must be faced. The first level of analysis is cost 
feasibility. Jf an alternative exceeds the available resources 
it cannot be considered. Simply, are the costs feasible? If 
no, adjust costs, or discard the project and consider an 
alternative. . 

However, don’t be hasty. Can trade-offs be made to make 
the costs feasible? For example, can you increase class size 
in order to hire a computer coordinator? Can planning time 
be used for staff development time? Can more software be 
substituted for more computers? Can more costs be 
deferred? Can you upgrade memory rather than buy more 
powerful computers? / 



TASK II: Are there any differences in the effectiveness 
and cost-effectiveness of feasible program alternatives? 

Cost-effectiveness takes information on the effectiveness of 
a program and compares it to the cost of the program in a 
systematic way. The basic technique has been to 
1) determine costs per unit of measure — i.e., cost per 
pupil, cost per work station, 2) derive effectiveness results, 
and 3) combine these results with cost per unit data to . 
determine the cost-effectiveness ratio of each alternative. 



Cost-effectiveness analysis assumes that: 1) the evaluation 
of effectiveness can be made separate from the evaluation 
of costs, 2) programs with similar or identical goals can be 
compared, and 3) a common measure of effectiveness can 
be used to assess them. 

Step 5: Determine the effectiveness of program 
alternatives. 

Choosing A ppropriate Effectiveness Measures . Cost 
analyst’s results and decision maker premises can only 
converge through increased use of multiple indicators of 
effectiveness. The use of single dimensions of effectiveness 
is inappropriate when one considers that many decision 
makers have multiple goals, and that all educational 
activities have multiple stakeholders (Windham, 1 990). 

Therefore, before conducting an analysis the cost analyst 
should identify appropriate measures of effectiveness held 
as critical by key decision makers. This improves the 
relevance of the quantitative data. 

At its simplest level, cost-effectiveness assesses total costs 
against unit costs. At its most sophisticated level, cost- 
effectiveness assesses outcomes in educational terms (e.g., 
student achievement, drop outs, etc.) in relationship to their 
costs. Two types of measures are emphasized in this paper 
because of their frequency of use, general availability from, 
standard data sources, and meaningfulness for policy 
analysis: unit costs and achievement costs. 

Unit costs. There is policy insight to be gained from the 
analysis of cost, even when effectiveness measures are 
not available. Analyzing by unit cost allows one to 
identify areas of potential inefficiencies by studying 
specific opportunities to improve costs per unit for a 
project. Often unit cost analysis is all the cost or effec- 
tiveness data permits. The first step in determining unit 
cost is to determine the unit or units of analysis (i.e. per 
student, classroom^ school, bus, work station). Then 
determine the number of units affected by the program, 
for example, 1,000 pupils, 55 buses, 75 work stations. 

Achievement costs . The purpose of the achievement cost 
* analysis is to identify whether a change in expenditure 
can lead to a change in achievement. The desired 
measure of outcomes is selected based on the value 
judgments of the key decision makers. If decision makers 
are interested in student opinions, they may find little 
value in information on enhanced mathematical skills. 



When decision makers have multiple subject areas of 
interest, they should analyze them individually rather than 
create an index. This permits decision makers to impose 
their values on interpretation of the results rather than the 
values of the presenter of the analysis. Once the achieve- 
ment measures are selected, one simply compares the cost 
. variations on the achievement measures. 

Decide on Data Entry Method and Enter Effectiveness 
Data . Gains on an effectiveness measure can be: 

• Predicted prior to adoption. Where predictions are 
being made, the person who is designing the project 
determines what change is expected to occur in the unit 
of measurement. For example, how many points of 
improvement are expected in a test score because of the 
anticipated expenditure; or how many students will not 
drop out? 

• Collected during the program. Cost-effectiveness 
can also be determined with the use of actual data that 
is collected during the program and/or after its comple- 
tion. This method of collection allows decision makers 
to analyze, evaluate and adjust the project prior to 
replication. 



TASK III: Calculate and Analyze Cost-Effectiveness 
Ratios 

Calculation . The cost-effectiveness ratio for each alterna- 
tive identifies how much it costs per unit to achieve a unit of 
gain for each program alternative. It is calculated by 
dividing the cost by the effectiveness for each achievement 
outcome, thereby yielding the cost to accomplish one unit of 
gain (for instance, one month of grade placement gain) per 
student. For example, for the dropout rate category a cost- 
effectiveness ratio of $3,500/1 indicates that preventing one 
student from dropping out would cost $3,500. Or, in the 
category of test scores, a ratio of $280/1 indicates that 
achieving a test score gain of 10 points costs $2,800 
($280x10). 

Analysis . Caution should be used in interpretation of ratios. 
It is generally assumed that cost-effectiveness increases the 
more cost units are affected, but not always. For example, 
when any cost factor is fixed (it does not increase in amount 
with enrollment), then the unit cost (per pupil) declines as 
enrollment increases. It is also possible that more effective 




strategies cost more (i.e. school size increases may cause 
costs per student to decrease and educational effectiveness 
to decline.) '• 

Step 6: Making Decisions with Cost-Analysis Results. 

The fact that computer-related activity is frequently “belief 
driven” rather than goal driven, argues for incorporating 
cost utility/feasibility analyses into the decision making 
process. These analyses evaluate outcomes' in terms of their 
subjective value to the decision maker, or in terms of 
resources available for acquisition. 

Of course, alternatives with the lowest costs relative to 
effects should have the highest priority for decisions. While 
it is the responsibility of decision makers to be informed of 
the quantitative findings, the final decision almost always 
comes down to a question of applying their own values and 
experiences in interpreting the available data. 

In the final analysis, all educational decisions, from the 
student-teacher interaction to the formulation of national 
policy, are cost utility decisions. 

Even when the decision makers accept the analysis data, 
they still must interpret the data for themselves in terms of 
the larger social or political systems within which they 
operate, and their own values. The best objective data will 
not eliminate the need for subjective judgments by decision 
makers. 

In addition to decision maker beliefs, there are four other 
issues that must be raised before making a final decision. 

1 . Are the results representative for the level of scale of the 
intervention that is appropriate? Some interventions have 
very different average costs per student when used for 30 
students than when used for 1,000. For example, a Com- 
puter-Assisted Instruction (CAI) system. that uses an 
expensive minicomputer might be very costly for 30 
students relative to stand-alone microcomputers. Thus, it is 
appropriate to make sure that the scale of use on which the 
cost-effectiveness ratio is based is pertinent to the decision 
making context. 

2. Can the results be extrapolated for effects that are 
beyond those found in the interventions.' For example, if 
$18 per student for alternative C generates an additional 
unit of effectiveness, would $180 generate 10 additional 
units of effectiveness? The answer to that question is that 
we do not know without further analysis of effectiveness ' 



under different intensities of resource use. Any generaliza- 
. tion beyond the range covered by the analysis must be 
based on additional evidence rather than being extrapolated. 

3. Can the magnitude of differences among alternatives be 
correctly interpreted through cost-effectiveness ratios? 
Decisions should not be based upon results that suggest 
only small differences in cost-effectiveness. For instance, 
when the differences are small, within a 10% - 20% range, it 
is probably best to use other criteria to make the decision. 
When estimated cost-effectiveness results are not markedly 
different, given the attainable precision in results, other 
considerations such as the ease of implementation, 
acceptability by staff, and experience with the intervention . 
should be used to make the decision (Levin, 1943). How- 
ever, in other cases, differences in cost-effectiveness among 
alternatives may be as large as 4, 5, or 10 to 1. In these 
cases, there are likely to be important gains in using 
resources for the most cost-effective of the alternatives. 

4. When applying cost-effectiveness analysis, how does its 
inherent bias toward short-term results affect the overall ' 
conclusions? This problem is pronounced with respect to 
decisions about new educational technologies. Evidence 
that CAI can be cost-effective in raising test scores is an 
important finding. But for many educators, the computer’s 
greatest promise is its potential to push back the traditional 
frontiers of cognition and learning and make possible the 
kinds of intellectual endeavor for which measures of 
achievement have not yet been invented. 
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